United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/747,107 


12/21/2000 


Lalitha Agnihotri 


US 000390 


1484 



7590 



12/29/2004 

PHILIPS ELECTRONICS NORTH AMERICAN CORP 
580 WHITE PLAINS RD 
TARRYTOWN, NY 10591 



EXAMINER 



USTARIS, JOSEPH G 



ART UNIT 



PAPER NUMBER 



2616 

DATE MAILED: 12/29/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO 90C (Rev 10/03) 



Office Action Summary 


Application ino. 

09/747,107 


Applicants; 

AGNIHOTRI etal 


Examiner 

Joseph G Ustaris 


Art Unit 

2616 





-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH (S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 
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DETAILED ACTION 
Response to Amendment 

1 . This action is in response to the amendment dated 12 July 2004 in application 
09/747,107. 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1-3, 7-9, 13-15, 19-21, 25-27, 29-31, and 34 are rejected under 35 
U.S.C. 102(e) as being anticipated by Ahmad et al. (US006263507B1). 

Regarding claim 1, Ahmad et al. (Ahmad) discloses a control device and system 
controller or "multimedia summary generator" for use in electronic devices such as 
televisions, a computer display monitor, or "video display system" (See Fig. 1; column 1 
lines 10-20 and column 2 lines 60-67). The control unit and system controller is able to 
summarize multiple television news programs or "video programs" within a graphical 
user interface (GUI) or "multimedia summary" (See Fig. 2A and 2B; column 4 lines 40- 
60). The control device and system controller extracts video images or "audio-video 
segments" (See column 16 line 55 - column 17 line 10) and text from a transcript (See 
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column 33 line 60 - column 34 line 10) and combines the extracted pieces to provide a 
summary of the television news broadcast within the GUI or "multimedia summary of 
said video program", wherein the summary criteria and GUI serve as the "audio-visual 
template" (See Fig. 2A, 2B, 3, and 4). Furthermore, Ahmad discloses the system can 
categorize the news programs and news stories or "identifying a domain of said video 
program" and selecting the news programs and stories video images and text 
"according to said domain" to be displayed to the user (See Fig. 2A, 201 ; Fig. 2B, 21 1 
and 215; column 3 lines 26-33 and column 15 lines 8-42). 

Regarding claims 2 and 3, the control device and system controller extracts video 
images that relates to the news stories or "topics" of the news program (See column 16 
line 55 - column 17 line 10), where inherently the news stories or "topics" are selected 
"according to the domain of said video program" when defined by the system and 
requested by the user (See Fig. 2B, 215: i.e. world news, national news, local news, 
etc;). The news programs have different stories, which are identified within the transcript 
by markers or "topic cue", wherein the system locates all the markers within the data or 
"determining a set of topic cues" (See column 23 lines 30-50). Each of the video images 
represents a news story within the GUI (See Fig. 2A and 2B), wherein the news story 
was located based on the markers or "topic cues". Furthermore, the control device and 
system controller executes this process, where inherently it is capable of executing 
computer instructions within a memory coupled thereto. 
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Claim 7 contains the limitations of claim 1 (wherein the control device and system 
controller are embodied along with a television, a computer display monitor, or "video 
display system") and is analyzed as previously discussed with respect to that claim. 

Claims 8 and 9 contains the limitations of claims 2, 3 T and 7 and is analyzed as 
previously discussed with respect to those claims. 

Claim 13 contains the limitations of claim 1 and is analyzed as previously 
discussed with respect to that claim. 

Claims 14 and 15 contains the limitations of claims 2, 3, and 13 and is analyzed 
as previously discussed with respect to those claims. 

Claim 19 contains the limitations of claim 1 and is analyzed as previously 
discussed with respect to that claim. Furthermore, Ahmad discloses that the process' 
can be executed via instructions within a computer readable medium encoded with one 
or more computer programs (See column 5 lines 5-10). 

Claims 20 and 21 contains the limitations of claims 2, 3, and 19 and is analyzed 
as previously discussed with respect to those claims. 

Regarding claim 25, the summary within the GUI has at least one video images 
of the news program, wherein the video images "relates to an identified domain of said 
video program" as discussed in claim 1 above (See Fig. 2A and 2B). 

Regarding claim 26, the summary within the GUI has at least one text portion of 
transcript (See Fig. 2A and 2B; column 16 line 55 - column 17 line 10; column 33 line 
60 -column 34 line 10). 
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Claim 27 contains the limitations of claims 2, 3, and 25 (wherein the GUI displays 
the summary information, including the video images representing each news story) and 
is analyzed as previously discussed with respect to those claims. 

Regarding claim 29, the GUI displays text summaries of the news program or 
"text from said video program", pictorial representations of the news program or "single 
frame", and video images or "video segment" (See Fig. 2A and 2B; column 16 line 55 - 
column 17 line 10). Furthermore, Ahmad discloses that the system is also capable of 
presenting information using audiovisual display or "audio-visual segment" (See column 
2 line 60 - column 3 line 1 5). 

Claim 30 contains the limitations of claims 2, 3, and 27 and is analyzed as 
previously discussed with respect to those claims. Furthermore, Ahmad discloses that 
the GUI has multiple video images or "a plurality of audio- visual segments representing 
each news story or "topic" (See Fig. 2b; column 16 line 64 - column 17 line 3). 

Regarding claim 31 , Ahmad discloses an information map region where the user 
can easily move freely among different news stories of the news program or "topic entry 
point". Each row allows the user to jump from one news story to another and would 
display the associated video images and text related to that story (See Fig. 2A and 2B; 
column 16 lines 3-55). 

Claim 34 contains the limitations of claims 30 and 31 and is analyzed as 
previously discussed with respect to those claims. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 4-6, 10-12, 16-18, 22-24, 28, 32, 33, and 35 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Ahmad et al. (US006263507B1) in view of Liou et al. 
(US006580437B1). 

Claim 4 contains the limitations of claims 1-3 and is analyzed as previously 
discussed with respect to those claims. However, Ahmad does not disclose a feature 
where the system can determine and identify subtopic cues and select a video image 
related to the subtopics to add to the GUI summary. 

Liou et al. (Liou) discloses a system for organizing videos based on closed- 
caption information. The system constructs a video table of contents. The system builds 
a tree defining the different stories or "topic cues" and the different speakers or 
"subtopic cues" associated with the stories based on the symbols or markers within the 
closed caption data or "determining a set of subtopic cues" and "identifying at least one 
subtopic cue". The system breaks the video up according to the closed-caption data and 
the tree is made up of video shots and text that are associated with each story and 
speaker (See Fig. 5, 6, and 9; column 1 lines 20-30, column 2 lines 40-65, column 6 
lines 35-55, column 7 lines 30-60). Therefore, it would have been obvious to one with 
ordinary skill in the art at the time the invention was made to modify the control device 
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and system controller disclosed by Ahmad to determine and identify speakers or 
"subtopic cues" within a news story and select a video image or shot related to the 
speaker to add to the GUI summary, as taught by Liou, in order to provide a more 
informative and detailed summary of the news program for the user. 

Regarding claim 5, the control device and system controller performs all the 
functions of the "domain identification application", "topic cue identification application", 
"subtopic cue identification application", and "audio-visual template identification 
application" as discussed in claims 1-4. 

Regarding claim 6, Ahmad discloses an information map region where the user 
can easily move freely among different news stories of the news program or "entry point 
for each topic". Each row allows the user to jump from one news story to another and 
would display the associated video images and text related to that story (See Ahmad 
Fig. 2A and 2B; column 16 lines 3-55). Furthermore, Liou discloses that the organized 
tree of the video table of contents allows users to jump directly to different speakers or 
"entry point for each subtopic" (See Liou Fig. 8 and 9). 

Claim 10 contains the limitations of claims 4 and 9 and is analyzed as previously 
discussed with respect to those claims. 

Claim 1 1 contains the limitations of claims 5 and 9 and is analyzed as previously 
discussed with respect to those claims. 

Claim 12 contains the limitations of claims 6 and 10 and is analyzed as 
previously discussed with respect to those claims. 



Application/Control Number: 09/747,107 Page 8 

Art Unit: 2616 

Claim 16 contains the limitations of claims 4 and 15 and is analyzed as 
previously discussed with respect to those claims. 

Claim 17 contains the limitations of claims 5 and 15 and is analyzed as 
previously discussed with respect to those claims. 

Claim 18 contains the limitations of claims 6 and 16 and is analyzed as 
previously discussed with respect to those claims. 

Claim 22 contains the limitations of claims 4 and 21 and is analyzed as 
previously discussed with respect to those claims. 

Claim 23 contains the limitations of claims 5 and 21 and is analyzed as 
previously discussed with respect to those claims. 

Claim 24 contains the limitations of claims 6 and 22 and is analyzed as 
previously discussed with respect to those claims. 

Claim 28 contains the limitations of claims 4 and 27 (wherein each speaker has 
one video shot related to the speaker (See Liou Fig. 8 and 9)) and is analyzed as 
previously discussed with respect to those claims. 

Claim 32 contains the limitations of claims 4 and 30 and is analyzed as 
previously discussed with respect to those claims. Furthermore, Official Notice is taken 
that it is well known to provide more than one video images or "plurality of audio-visual 
segments" for each speaker of a program. Therefore, it would have been obvious to 
modify the control device and system controller disclosed by Ahmad in view of Liou to 
provide more than one video images in order to provide a more detailed visual summary 
of where the speaker is in association with the story. 
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Claim 33 contains the limitations of claims 6, 31 , and 32 and is analyzed as 
previously discussed with respect to those claims. 

Claim 35 contains the limitations of claims 32 and 33 and is analyzed as 
previously discussed with respect to those claims. 

Claims 36-38 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ahmad et al. (US006263507B1) in view of Name-It Feature Article and Hoarty et al. 
(US005093718A). 

Claim 36 contains the limitations of claim 13 and is analyzed as previously 
discussed with respect to that claim. However, Ahmad does not disclose a feature 
where it obtains an image of a face of a person, verifies the identity, and adds the image 
of the face to the GUI summary. 

The article Name-It discloses a method for matching faces in a news video with 
their name or "identity". The method extracts the face of a person from a video and the 
name of the person from the transcript of the video. Then it compares the collected 
information to the face similarity information to confirm that the name is matched to the 
correct face (See page 22 column 1 and 2, page 25 column 1 and 2). Therefore, it 
would have been obvious to one with ordinary skill in the art at the time the invention 
was made to modify the control device and system controller disclosed by Ahmad to 
extract an image of a face of a person and confirm the persons name, as taught by the 
article Name-It, in order to provide a means of correctly identifying the people on the 
video so that the user is aware of who is presenting the story. 
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Hoarty et al. discloses an interactive home information system where users can 
obtain information about TV listing for the month. The system provides images of actors 
or scenes from movies and TV shows within the information or "summary" (See column 
7 lines 20-30). Therefore, it would have been obvious to one with ordinary skill in the art 
at the time the invention was made to modify the control device and system controller 
disclosed by Ahmad to add the image of the face to the GUI summary, as taught by 
Hoarty et al., in order to provide a more detailed summary of the story and who is 
presenting the story in one convenient screen. 

Regarding claim 37, the system confirms the identity of the person by using the 
face similarity information of the person and the name from the transcript as discussed 
in claim 36. 

Claim 38 contains the limitations of claims 36 and 37 and is analyzed as 
previously discussed with respect to those claims. Furthermore, Official Notice is taken 
that it is well known to use voice recognition to identify a person. Therefore, it would 
have been obvious to one with ordinary skill in the art at the time the invention was 
made to modify the control device and system controller disclosed by Ahmad in view of 
Name-It to identify a person using voice recognition in order to provide a more accurate 
means of correctly identifying the person. 

Response to Arguments 

4. Applicant's arguments filed 12 July 2004 have been fully considered but they are 
not persuasive. 
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Applicant argues with respect to claims 1 , 7, 13, 1 9, 25, and 34 that Ahmad does 
not teach or suggest identifying the domain of the program and selecting portions of the 
transcript and audio-video segments of the program according to its domain. However, 
such limitations are disclosed by Ahmad as discussed in the claim rejection above. 

It is noted that that the previous Office Action contained Official Notice 
statements which have not been traversed by the applicant. Therefore, these 
statements have therefore been taken as admissions of prior art as dictated by MPEP § 
2144.03. 

Applicant is reminded that although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. See In re 
Van Geuns, 988 F.2d 11 81 , 26 USPQ2d 1057 (Fed. Cir. 1993). 

Conclusion 

5. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Joseph G Ustaris whose telephone number is 703-305- 
0377. The examiner can normally be reached on M-F 7:30-5PM; Alternate Fridays off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew I Faile can be reached on 703-305-4380. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 





December 13, 2004 



SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



